1. Field of the Invention
The disclosures herein generally relate to an image processing apparatus, an image processing method, and an imaging system, more specifically, an image processing apparatus, an image processing method, and an imaging system that piece multiple input images together.
2. Description of the Related Art
An entire celestial sphere imaging system has been known that uses multiple wide-angle lenses such as fisheye lenses or superwide-angle lenses to capture an omniazimuth (referred to as “entire celestial sphere” below) image at once. The entire celestial sphere imaging system generates an entire celestial sphere image by projecting images from the lenses onto sensor surfaces and piecing the images together by image processing. It is possible to generate such an entire celestial sphere image by using, for example, two wide-angle lenses having field angles greater than 180°.
In the image processing, distortion correction and projective transform are applied to partial images captured by the lens optical systems, based on a predetermined projection model and considering distortion from an ideal model. Then, the partial images are pieced together using overlapped parts included in the partial images, to generate a single entire celestial sphere image. In the process of piecing the images together, joint positions of overlapped objects are detected by using pattern matching in the overlapped parts of the partial images.
However, for the conventional joint position detection technology using pattern matching, it is difficult to detect appropriate joint positions if a region to be matched is a flat image or has few characteristics where the same pattern is repeated. Therefore, the partial images cannot favorably be pieced together, which may reduce the quality of the obtained entire celestial sphere image.
Various technologies have been known that piece captured multiple partial images together using multiple cameras. For example, Japanese Laid-open Patent Publication No. 2001-148779 (Patent Document 1) discloses an image processing apparatus that has an object to synthesize an image with high precision by avoiding an error of pattern matching caused when an inappropriate matching region is used when performing the pattern matching. The conventional technology in Patent Document 1 is configured to determine whether a matching region extracted by a matching region extraction unit is appropriate, and if the matching region is inappropriate, to execute extraction again. It is also configured to determine whether a matching result by a pattern matching unit is appropriate, and if the matching result is not appropriate, to execute extraction of a matching region again.
To improve precision of image synthesis, the conventional technology in Patent Document 1 removes in advance regions inappropriate as matching regions, for example, a region of all white pixels or black pixels, or a region including a continuous line in the vertical direction, the lateral direction, or an oblique direction before performing pattern matching.
However, the conventional technology in Patent Document 1 is a technology that executes extraction of a matching region again if a matching region is inappropriate. Therefore, it is still difficult to determine an appropriate joint position for a matching region that is determined as inappropriate.